just supplementary..

User Stories

S1: The model's broken!

Validate RDFS (maybe also OWL) models The basic issue here is to ensure that the right kind of information is given for each property (or class) in the model, for example, to require that each property has to have a domain, or that classes have to be explicitly stated to be under some decomposition. Input data: the RDF representation of an RDFS (or OWL) ontology Input ontology: the ontology that represents RDFS (or OWL) syntax

Requires the ability to check whether certain information is given/available for a property or class.

S2: What's the name of that person?

For a tool to build list of names for named entity resolution of people to work correctly, every person has to have one or more names specified, each of which is a string. Constraints can be used to verify that particular sets of data have such names for each person.

Requires the ability to check the cardinality of properties as well as the type of its values.

Related to: S8,S11,S23, S37

S3: Communicating back to users, kindly

Rather than rejecting or having yes/no, and discouraging users and rejecting a lot of data, have a number of responses that inform users of ways they could improve their data, while still accepting all but the truly unusable data. This requires levels of "validation".

Requires the ability to return responses appropriate to the condition, not just "pass/fail."

S4: Issue repository

An LDP Container accepts an IssueShape with a status of "assigned" or "unassigned". The LDP Container is an interface to a service storing data in a conventional relational database. The shapes are "closed" in that the system rejects documents with any triples for which it has no storage. The shapes validation process (initiated by the receiving system or a sender checking) rejects any document with "extraneous" triples.

Any node in the graph may serve multiple roles, e.g. the same node may include properties for a SubmittingUser and for an AssignedEmployee.

Later the issue gets resolved and is available at without acquiring new type arcs. The constraints for are different from those for Issues at

Requires the ability to associate more than one shape to the same graph or node.

S5: Closed-world recognition (EPIM ReportingHub)

EPIM Project - petroleum operators on the Norwegian continental shell need to produce environment reports of what chemicals were dumped into the sea and gases to the air. There is a need for access rules on what operators can see what data from what oil and gas fields, and for complex constraints to run during import of XML files. SPIN was used to represent and evaluate those constraints.

This is an example of very complex constraints that require many features from SPARQL to represent model-specific scenarios, including the comparison of incoming values against a controlled fact base, transformations from literal values to URIs, string operations, date comparisons etc. User-defined SPIN functions were used to make those complex queries maintainable.

Details: EPIM ReportingHub

Requires the expressibility of complex constraints that include e.g. value transformations, string operations, date comparisons, etc.

S6: Closed-world recognition for e.g. for partial ontology import

Importing all of an ontology is not always a good practice. When an ontology is imported it is often the case that many concepts and properties will be irrelevant to the needs at hand. In addition transitive imports can lead to increased "Ontology Glut". An increasingly popular practice is to not do any imports but to explicitly declare the use of non-imported resources with rdfs:definedIn to provide the provenance to the authoritative defining point of the resource. Alternatively some way to constrain imports to avoid ontology glut might be useful.

SPIN currently uses owl:imports to include other graphs. If no owl:imports statement is present, then the engine will not execute constraints stored in the remote schema. It is perfectly fine to have local copies of classes and properties defined elsewhere, without requiring the full contract. This is a common scenario in controlled environments, not the full Web.

still has open issue

Must be able to define and/or import only those parts of an ontology that are needed for the validation.

S8: Checking RDF node type

It is often necessary or desirable to check whether certain property values (or more general: RDF nodes) are of a specific node type (IRI, BlankNode or Literal and all combinations thereof). Often the intention is to state that a given property shall only have IRIs but not BlankNodes.

Two examples from the VOID namespace, void:dataDump and void:exampleResource declare an rdfs:range of rdfs:Resource, but the intention is to only support IRI resources.

The DCAT namespace has similar examples, where only IRI nodes are permitted: dcat:landingPage, dcat:accessURL and dcat:downloadURL. The declared ranges are foaf:Document or rdfs:Resource. The foaf:Document case is interesting, shows that people might want to specify both the value's class (foaf:Document) and node type (URI only).

SPARQL includes the built-ins isIRI, isBlank, isLiteral for those checks.

Requires the possibility to constrain the value of a property. E.g. check whether it is an IRI, a literal, a blank node, or some combination of those.

Related to: S2,S11,S23, S37

S9: Contract time intervals

OMG time ontology adopted by FIBO. end date *exists* but may not be specified. Some contracts (bonds) have an end date.

still has open issue

Validation must allow for (momentarily) unspecified values. For example, an end date may be assumed but is not specified at this time.

S10: card >= 0

Mention a property in a card>= 0 restriction, just to indicate an expectation that it will (or might) be there without requiring that it be there.

(ericP: may I replace this with a story with requirements for different cardinalities, including fixed cardinalities > 1 (2 comes up a lot, e.g. two biological parents)? I propose:)

Clinical data requires specific cardinality constraints, e.g.

  • zero or one (optional) birth date.
  • zero or more lab tests.
  • one active patient marker.
  • one or more emergency contact.
  • two biological parents.

still has open issue

Must support "optional" properties through the use of cardinality >=0.

S11: Model-Driven UI constraints

Need to have constraints provide model-driven validation of permissible values in user interfaces. A number of solutions and applications have been deployed which use SPIN to check constraints on permissible values to user interfaces. This overcomes the software debt that comes from using javascript that can readily become out-of-sync with the underlying models.

The major requirement here is a declarative model of:

  • which properties are relevant for a given class/instance?
  • what is the value type of those properties?
  • what is the valid cardinality (min/maxCount)?
  • what is the interval of valid literal values (min/maxValue)?
  • any other metadata typically needed to build forms with input widgets.

A meta-requirement here is to be able to make use of the information above without having to run something like SPARQL queries, i.e. the model should be sufficiently high level so that all kinds of tools can use that information. However, at the same time there are many advanced constraints that need to be validated (either on server or client) before a form can be submitted. These constraints are not necessarily "structural" information, but rather executable code that returns error messages.

Requires the ability to declare and constrain permitted values for properties, as well as their cardinalities, in an abstract and "high-level" fashion.

Related to: S2,S8,S23, S37

S12: App Interoperability

For example, cimba.co acts as a decentralized twitter, operating over LDP. For another app to interoperate, it needs to know what data shapes cimba reads and write. This is currently documented with diagrams and sparql templates. The SPARQL is fairly complex and hard to read, and it seems like another language might make it easier to write interoperable programs.

still has open issue

The solution shall be usable to document applications, like APIs, for application interoperability. It should be possible to use the shape definition between applications to manage data exchange.

S13: Specification and validation of metadata templates for immunological experiments

Systems Biology is playing an increasingly important role in unraveling the complexity of human immune responses. A key aspect of this approach involves the analysis and integration of data from a multiplicity of high-throughput immune profiling methods to understand (and eventually predict) the immunological response to infection and vaccination under diverse conditions. To this end, the Human Immunology Project Consortium (HIPC) was established by the National Institute of Allergy and Infectious Diseases (NIAID) of the US National Institutes of Health (NIH). This consortium generates a wide variety of phenotypic and molecular data from well-characterized patient cohorts, including genome-wide expression profiling, high-dimensional flow cytometry and serum cytokine concentrations. The adoption and adherence to data standards is critical to enable data integration across HIPC centers, and facilitate data re-use by the wider scientific community.

In collaboration with ImmPort, we have developed a set of spreadsheet-based templates to capture the metadata associated with experimental results such as Flow Cytometry results and Multiplex Bead Array Assay (MBAA) results. These templates contain metadata elements that are either required or optional, but importantly, define the value of the field to specific datatypes (e.g. string, integer, decimal, date) that may be restricted by length or to a regular expression pattern, and limited to specific categorical values or terminology trees/class expressions of a target ontology, especially those drawn from existing ontologies such as Cell Ontology (CL) and Protein Ontology (PO). Once filled out, these spreadsheets are programmatically validated. The values are then stored in a database and are used to power web applications and application programming interfaces.

Given the rapid change in the kinds of experiments performed and the evolving requirements concerning relevant metadata, it is crucial that a language to define these metadata constraints enable us to define different sets of metadata fields and values sets in a modular manner. In addition to HIPC, there are other immunology consortia that might involve different requirements as to how data templates should be defined according to specific needs. It should be relatively straightforward to substitute one set of shape expressions for another. It is also important that the shapes themselves are versioned and the results of validation record the version of the shape expression. It should be possible to validate data using any set of developed shapes.

Ideally, the shapes language should be readable by computers in order to automatically generate template forms with restriction to specified values. Moreover, libraries and tools to construct and validate templates and their instance data should be readily available.

Requires the possibility to define shapes for a specific node in a modular manner, i.e. defining different sets of metadata fields and value sets.

Requires the availability of version information of shapes thus, the results of validation shall record the version of the triggered shape expression.

Related to(regarding overall constraint requirements): S2,S8,S23, S37

S14: Object Reconciliation

As an aid in data integration activities, it would be nice if shapes could flexibly state conditions by which to check that identity of objects has been correctly recorded; that is, check conditions under which 2 objects in a KB should explicitly represent the same real-world thing. For example (movies domain), I'd like to say:

  • if source1.movie.title is identical to source2.film.title AND source1.movie.release-date.year is close (say, < 2 years difference) to source2.film.initial-release then it should be stated that they are the same movie OR
  • if source1.movie.directors has the same set of values as source2.film.directed-by AND source1.movie.title is highly similar to source2.film.title then it should be stated that they are the same movie OR
  • ...

still has open issue

The ability to define conditions that can be used to determine if two separate descriptions represent the same real world object.

S15: Validation of Dataset Descriptions

Access to consistent, high-quality metadata is critical to finding, understanding, exchanging, and reusing scientific data. The W3C Health Care and Life Sciences Interest Group (HCLSIG) has developed consensus among participating stakeholders on key metadata elements and their value sets for description of HCLS datasets. This specification, written as a W3C note, meets key functional requirements, reuses existing vocabularies, is expressed using the Resource Description Framework (RDF). It provides guidance for minimal data description, versioning, provenance, statistics. We would like to use RDF Shapes to specify these constraints and validate the correctness of HCLS dataset descriptions.

The specification defines a 3 component model for summary,versioning, and distribution-level descriptions. Each component has access to a specific set of metadata elements and these are specified as MUST, SHOULD, MAY, and MUST NOT. As such there are different conformance criteria for each level. Metadata values are either unrestrained rdfs:Literals, constrained rdfs:Literals, URIs with a specified URI pattern, or instances of a specified URI-identified type, or a disjunction of URI-specified types.

Cardinalities and ranges are covered by all existing proposals, so I guess the interesting bit here is how to represent that certain constraints only apply in certain contexts ("levels: summary, version, distribution").

Requires the functionality to restrict application of constraints to certain contexts.

Requires expressibility of cardinality constraints and property value restrictions.

S16: Constraints and controlled reasoning. We need both!

A use-case we were facing recently and have discussed in [1], was revolving around the integration of distributed configurations (i.e. object-oriented models) with RDFS and SPARQL.

In this particular use-case we had to assume both Unique Name Assumption (UNA) and Closed World Assumption (CWA) for our ontologies, since the models (i.e. configurations) those ontologies were derived from were generated by product configurators which impose both UNA and CWA. Since neither RDFS or OWL impose UNA/CWA we had to come up with some workarounds which were basically:

  • UNA 2.0: all URIs are treated as different, unless explicitly stated otherwise by owl:sameAs (UNA 2.0 because in general, if two URIs are different and the ontology they are contained in is assumed to obey the UNA then they cannot be connected via owl:sameAs).
  • CWA: we assumed to know every existing individual of local configurations and directly connected individuals from other local configurations, thus an absence of a certain individual in the local configuration means that it does not exist.

    As mentioned earlier, we used SPARQL to perform query tasks on the global schema as well as to check simple integrity constraints by translating e.g. cardinality restrictions into ASK queries.

    One major problem which arose based on our workaround to impose UNA was, that SPARQL is unaware of the special semantics of owl:sameAs. Which means that especially if one wants to use counting aggregates, one usually wants to count the number of real-objects and not the number of URIs referring to it. As an example we defined two SPARQL queries which should count the number of subnets of a certain system:

    SELECT (COUNT(DISTINCT ?subnet) AS ?numberofsubnets)
    	WHERE {
    	?subnet a ontoSys:Subnet .
    	}
    	# result: numberofsubnets = 3 
    					        
    	SELECT (COUNT(DISTINCT ?subnet) AS ?numberofsubnets)
    	WHERE {
    	 ?subnet a ontoSys:Subnet .
    	 # first subquery
    	 { SELECT ?subnet ?first
    		WHERE {
    		 ?subnet ((owl:sameAs|^owl:sameAs)*) ?first .
    		 OPTIONAL {
    		 ?notfirst ((owl:sameAs|^owl:sameAs)*) ?first .
    			FILTER (STR(?notfirst) < STR(?first))}
    			FILTER(!BOUND(?notfirst))}
    	      }
    		}
    	# result : numberofsubnets
    				

    Obviously Example 2 is way more ugly than Example 1, especially due to some nasty path expressions which are necessary to traverse through potential owl:sameAs chains. Other approaches such as replacing those chains with pivot-identifiers in a potential pre-processing step are not feasible since we actually want to keep the different identifiers separate in the data for particular use-cases.

Requires support of unique name assumption, such that each unique IRI is assumed to represent a unique entity.

Requires possibility to encapsulate verbose constraint definitions into macros thus, allow their reuse in other shapes as well as increase readability of shape expressions.

S17: Specify subsets of data

Have a lightweight way to refer to a part of a data set, based on the shapes. This could be used for entitlements as well ("you can see AML/KYC shape for this class", "You can only see the identification shape for this class")

still has open issue

..

S18: Scope of Export

Starting from a given KB object (individual), I want to export a bunch of related stuff. Use shapes to specify the paths / conditions by which the stuff to be exported can be selected.

still has open issue

..

S19: Query Builder

Various tools are contributing data to a triple store. A Query Builder wants to know the permitted or likely shapes of the data over which the generated queries must run, so that the end user can be presented with a nice interface prompting for likely predicates and values. Since the data is dynamic, this is not necessarily the same as the shape that could be reverse engineered from the existing data. The Query Builder and the data-producing tools are not provided by the same team - the Query Builder team has very limited control over the data being produced. The source of the data might not provide the necessary shape information, so we need a way for the Query Builder team (or a third party) to be able to provide the shape data independently. See also Ontology-Driven Forms and S11.

still has open issue

..

S20: Creation Shapes

A client creating a new resource by posting to a Linked Data Platform Container [2] wants to know the acceptable properties and their values, including which ones are mandatory and which optional. Note that this creation shape is not necessarily the same as the shape of the resource post-creation - the server may transform some values, add new properties, etc. [2] http://www.w3.org/TR/ldp/#ldpc

See the ongoing discussion at http://lists.w3.org/Archives/Public/public-data-shapes-wg/2014Nov/0160.html with hints at a solution based on named graphs. Other solutions with stand-alone shapes have been proposed as well as an option to select constraints based on decorations (annotations)

still has open issue

..

S21: SKOS Constraints

The well-known SKOS vocabulary defines constraints that are outside of the expressivity of current ontology languages. They can be expressed using SPARQL built-ins, e.g. via SPIN. Examples include:

  • make sure that a resource has at most one preferred label for a given language
  • preferred labels and alternative labels must be disjoint
Details: SKOS Constraints

Requires the possibility to define complex constraints including ones on property/value pairs.

Related to: Cardinality constraints, constraints over properties of the same node.

S22: RDF Data Cube Constraints

The Data Cube Vocabulary provides a means to publish multi-dimensional data, such as statistics, on the web in such a way that it can be linked to related data sets and concepts. While the bulk of the vocabulary is defined as an RDF Schema, it also includes integrity constraints:

Each integrity constraint is expressed as narrative prose and, where possible, a SPARQL ASK query or query template. If the ASK query is applied to an RDF graph then it will return true if that graph contains one or more Data Cube instances which violate the corresponding constraint.

Using SPARQL queries to express the integrity constraints does not imply that integrity checking must be performed this way. Implementations are free to use alternative query formulations or alternative implementation techniques to perform equivalent checks.

#Every qb:DataStructureDefinition 
#must include at least one declared measure
	ASK {
	?dsd a qb:DataStructureDefinition .
	FILTER NOT EXISTS {
    ?dsd qb:component 
       [qb:componentProperty 
          [a qb:MeasureProperty]] }
					}
				

Requires support of RDF Data Cube Integrity Constraints

Related to: Cardinality constraints, constraints over properties of the same node, property value restrictions.

S23: schema.org Constraints

Developers at Google have created a validation tool for the well-known schema.org vocabulary for use in Google Search, Google Now and Gmail. They have found that what may seem like a potentially infinite number of possible constraints can be represented quite succinctly using existing standards like the SPARQL query language and serialized as RDF.

  • On schema:Person: Children cannot contain cycles, Children must be born after the parent, deathDate must be after birthDate
  • On schema:GeoCoordinates: longitude must be between -180 and 180, latitude between -90 and 90
  • On various: email address must match a certain regular expression
  • On schema:priceCurrency, currenciesAccepted: Currency code must be from a given controlled vocabulary
  • On schema:children, colleagues, follows, knows, parents, relatedTo, siblings, spouse, subEvents, superEvents: Irreflexitity

Solution from the Google Paper (JSON-LD), replacing boardingTime with departureTime:

					#Boarding passes will only be shown in Google 
                   #Now for flights which occur at a future date: 
					{
					    "@context": {...},
					    "@id": "schema:FlightReservation",
					    "constraints": [{
					       "context": "schema:reservationFor",
					       "constraint": "ASK WHERE 
                            {?s schema:departureTime ?t. 
                             FILTER(?t > NOW())}",
					       "severity": "warning",
					       "message": "A future date is required 
                            to show a boarding pass.",
					    }]
					}
					

Requires support of schema.org constraints.

Related to: S2,S8,S11, S37

Related to: Property value restrictions.

S24: Open Content Model

Suppose there is a need to integrate similar information from multiple applications and that the application owners have gotten together and defined an RDF representation for this information. However, since the applications have some differences, the application owners can only agree on those data items that are common to all applications. The defined RDF representation includes the common data items, and allows the presence of other undefined data items in order to accommodate differences among the applications. In this situation, the RDF representation is said to have an open content model. In fact, one of the attractive features of RDF technology is that it readily enables open content models.

For example, the OSLC Change Management (CM) specification specifies a very minimal representation for Change Requests (e.g. bug reports). A large software development organization may use several different change management tools, e.g. Bugzilla, Jira, and ClearQuest, each with their own proprietary resource format. The OSLC CM specification provides a way to process change management information in a uniform way, independently of the tool that hosts it. However, there may also be interesting differences in the type of information hosted by each tool. OSLC therefore specifies an open content model which allows implementations to extend the base representation with additional content. This content is represented as additional RDF properties on the resources. Furthermore, it is very common for change management tools to partition their resources into defined projects which restrict who can access the resources and which define custom attributes on the resources. Here the term custom attribute refers to an attribute that is not defined out-of-the-box in the tool. The tool administrators customize the tool by defining custom attributes, typically on a per-project basis. For example, one project might add a customer reference number while another might add a boolean flag indicating if there is an impact to the online documentation. These custom attributes also appear as additional RDF properties of the resources.

OSLC specifications typically define one or more RDF types. For example, the RDF type for change requests is oslc_cm:ChangeRequest where the prefix oslc_cm is . The RDF representation of an OSLC change request contains a triple that defines its type as oslc_cm:ChangeRequest, triples that define RDF properties as described in the OSLC CM specification, and additional triples that correspond to tool-specific or project-specific custom attributes. Note that the addition of custom attributes does not require the definition of a new RDF type. Furthermore the RDF properties used to represent custom attributes may come from any RDF vocabulary. In fact, tool administrators are encouraged to reuse existing RDF properties rather than define synonyms.

Since the shape of a resource may depend on the tool that hosts it, or the project that hosts it within a tool, but the RDF type of the resource may not depend on the tool or project, there is in general no way to navigate to the shape given only its RDF type. The OSLC Resource Shapes specification provides two mechanisms for navigating to the appropriate shape. First, the RDF property oslc:resourceShape where oslc: is may be used to link a tool or project description to a shape resource. Second, the RDF property oslc:instanceShape may be used to link a resource to its shape.

See Open Content Model Example for a detailed example.

Requires the possibility to address a resource graph based on criteria unrelated to its rdf:type. This can be a general context, or a specific application function.

Related to: S4

S25: Primary Keys with URI Patterns

It is very common to have a single property that uniquely identifies instances of a given class. For example, when you import legacy data from a spreadsheet, it should be possible to automatically produce URIs based on a given primary key column. The proposed solution here is to define a standard vocabulary to represent the primary key and a suitable URI pattern. This information can then be used both for constraint checking of existing instances, and to construct new (valid) instances. One requirement here is advanced string processing, including the ability to turn a partial URI and a literal value into a new URI.

Details: Primary Keys with URI Pattern

Requires The ability to create IRIs from non-IRI identifiers.

S26: rdf:Lists and ordered data

Can we express validating rdf:Lists a in our framework? This is more than just a stresstest but a variation of this can be used to check whether all members of a list have certain characteristics.

Libraries have a number of resources that are issued in ordered series. Any library may own or have access to some parts of the series, either sequential or with broken sequences. The list may be very long, and it is often necessary to display the list of items in order. The order can be nicely numerical, or not. Another ordered list use case is that of authors on academic journal articles. For reasons of attribution (and promotion!), the order of authors in article publishing can be significant. This is not a computable order (e.g. alphabetical by name). There are probably other cases, but essentially there will definitely be a need to have ordered lists for some data. Validation could be: the list must have a beginning and end; there can be/cannot be gaps in the list.

Details: rdf:List Stresstest

Requires the possibility to define ordered and unordered lists of properties, including attributes like begin_element, end_element, etc.

S27: Relationships between values of multiple properties

Cultural heritage data is created in a distributed way, so when data is gathered together in a single aggregation, quite a bit of checking must be done. One of the key aspects of CH data is the identification of persons and subjects, in particular relating them to historical contexts. For persons, a key context is their own birth and death dates; for events, there is often a date range representing a beginning and end of the event. In addition, there are cultural heritage objects that exist over a span of time (serial publications, for example). In each of these cases, it is desirable to validate the relationship of the values of properties that have temporal or other ordered characteristics.

Details: Relationships between values of different properties

Requires ability to perform comparisons on the values in selected sets of properties. For example, to compare the value of properties representing birth date and death date to validate that birthdate precedes death date. Similar tests may be needed within workflows, for example to check that step one is completed before step two.

Related to: Constraints over properties of the same node.

S28: Self-Describing Linked Data Resources

In Linked Data related information is accessed by URI dereferencing. The information that is accessible this way may represent facts about a particular resource, but also typing information for the resource. The types can themselves be used in a similar way to find the ontology describing the resource. It should be possible to use these same mechanisms to find constraints on the information provided about the resource.

For example, the ontology could include constraints or could point to another document that includes constraints. Or the first document accessed might include constraints or point to another document that includes constraints.

DCMI story: For some properties there is a requirement that the value IRI resolve to a resource that is a skos:Concept. The resource value is not limited to a particular skos:Concept scheme.

1. The validation language must be able to make use of information obtained through dereferencing of the property IRI.*

2. The validation language must be able to define validation for information received from a dereferencing of the value IRI, e.g. that the value is a member of a skos:ConceptScheme.

* #1 strikes me as problematic since what is returned will be RDF/OWL with open world semantics.

S29: Describing interoperable, hypermedia-driven Web APIs (with Hydra)

Hydra is a lightweight vocabulary to create hypermedia-driven Web APIs. By specifying a number of concepts commonly used in Web APIs it enables the creation of generic API clients. The Hydra core vocabulary can be used to define classes and "supported properties" which carry additional metadata such as whether the property is required and whether it is read-only.

This feels very similar to the OSLC Resource Shapes story and uses similar constructs. It is also possible to express the supported properties as a SPIN constraint check, as implemented here: http://topbraid.org/spin/spinhydr

Requires the possibility to define a set of routines or concepts that will fulfil commonly required validation tasks, with perhaps some selectable options.

S30: PROV Constraints

The PROV Family of Documents defines a model, corresponding serializations and other supporting definitions to enable the inter-operable interchange of provenance information in heterogeneous environments such as the Web. One of these documents is a library of Constraints which defines valid PROV instances. The actual validation process is quite complex and requires a normalization step that can be compared to rules. Various implementations of this validation process exist, including a set of SPARQL INSERT/SELECT queries sequenced by a Python script, an implementation in Java and in Prolog. Stardog also defines an "archetype" for PROV, which seems to be implemented in SPARQL using their ICV engine.

Requires support of PROV Constraints.

Requires a mechanisms to define rules within shape definitions.

S31: LDP: POST content to Container of a certain shape

Some simple LDP server implementations may be based on lightweight app server technology and only deal with JSON(-LD) and Turtle representations for their LDP RDF Sources (LDP-RS) on top of an existing application, say Bugzilla. As a client implementer, I may have a simple JavaScript application that consumes and produces JSON-LD. I want to have a way to programmatically provide the end-user with a simple form to create new resources and also a way to potential auto-prefill this form based on data from current context.

LDP defines some behavior when a POST fails to a ldp:Container, by outlining expected status codes and additional hints that could be found in either the response body of the HTTP POST request or a response header (such as: Link relation of "http://www.w3.org/ns/ldp#constrainedBy". A client can proactively request headers (instead of trying the POST and it fails) by performing an HTTP HEAD or OPTIONS request on the container URL and inspecting the link relation for "constrainedBy". Typical constraints are: a) not necessarily based on type b) sometimes limited to the action of creation and may not apply to other states of the resource.

Current gap is whatever is at the end of the "constrainedBy" link, could be anything: HTML, OSLC Resource Shapes, SPIN. The LDP WG discussed a need to have something a bit more formalized and deferred making any recommendation looking to apply these requirements unto the Data Shapes work.Once it matures, and meets the requirements, LDP could provide a recommendation for it then.

covered in S11?

S32: Non-SPARQL based solution to express constraints between different properties

Consider the case of clients consuming RDF resources, interfacing with an LDP container, needs to work in a disconnected mode (the client being a Workers mobile device where the work zone has no connectivity). The client needs to allow workers to create entries locally in the device to mark completion of different stages of the work. These entries will get synched up with the LDP container at a later time, when the device gets connectivity back. Prior to that, when the client is in disconnected mode, the client software needs to perform a range of validations on the users entries to reduce the probabilty of an invalid entry.

In addition to the basic data type/required/cardinality "stand alone" validations, the client needs to validate constraints between different properties:

  • start time less than end time
  • if end time is not specified, the status of the "work" should be "In Progress"
  • if status is "Complete" end time is required.
The client side does not have access to any triple store/LDP container. If these validations can be expressed in a higher level language which makes it simpler for clients to implement them constraint systems will be useful in more places.

Expresses the requirement to be able to define constraints over more than one property. E.g., value of property start time must be less than value of property end time.

Those interdependencies between properties of the same RDF node should be expressible in a higher level language.

Related to: S27

S33: Structural validation for queriability

Patient data (all data) is frequently full of structural errors. Statistical queries over malformed data leads to misinterpretation and inaccurate conclusions. Shapes can be used to sequester well-formed data for simpler analysis.

Consider a schema where a medical procedure should have no more than one outcome. Accidental double entry occurs when e.g. a clinician and her assistant both enter outcomes into the database:

					 _:Bob :hadIntervention [
					    :performedProcedure [ a bridg:PerformedProcedure ;
					                          :definedBy [ :coding term:MarrowTransplant ; :location terms:Manubrium ] ];
					    :assessmentTest     [ a bridg:PerformedObservation ;
					                          :definedBy [ :coding term:TumorMarkerTest ; :evaluator  ] ;
					                          :result    [ :coding term:ImprovedToNormal ; :assessedBy clinic:doctor7 ],
					                                     [ :coding term:ImprovedToNormal ; :assessedBy clinic:doctor7 ]
                        ]
                    ] .
					
The obvious SPARQL query on this will improperly weight this as two positive outcomes:
					 SELECT ?location ?result (COUNT(*) AS ?count)
					 WHERE {
					   ?who :hadIntervention [
					      :performedProcedure [ :definedBy [ :coding term:MarrowTransplant ; :location ?location ] ];
					      :assessmentTest     [ :definedBy [ :coding term:TumorMarkerTest ] ;
					                            :result    [ :coding ?result ] ]
					   ]
					 } GROUP BY ?result ?location
					
(This is a slight simplification for the sake of readability. In practice, an auxilliary hierarchy identifies multiple codes as positive outcomes, e.g. term:ImprovedToNormal and term2:ClinicalCure, but the effect is the same as described here.)

Requires the ability to perform structural validation over RDF data. Being able to select subsets of data related to an RDF node and thus, define a well-formed/cleansed representation of that node (which is represented as shape), allows to improve the quality of data as well as its queriability.

S34: Large-scale dataset validation

A publisher has a very large RDF Database (millions - billions triples) and wants to define multiple shapes for the data that will be checked at regular intervals. To make this process effective 1) validation must run within a reasonable time-span and 2) it must be possible to determine just what violations were found, i.e., just a TRUE/FALSE result is inadequate.

Basically a repetition of S3 with additional requirements regarding the validation performance.

S35: Describe disconnected graphs

In general, the RDF representation of an information resource may be a disconnected graph in the sense that the set of nodes in the graph may be partitioned into two disjoint subsets A and B such that there is no undirected path that starts in A and ends in B. The shape language must be able to describe such graphs. For example, consider the following JSON-LD representation of the Access Context List resource specified in OSLC Indexable Linked Data Provider Specification V2.0:

					{
					  "@context": {
					    "acc": "http://open-services.net/ns/core/acc#",
					    "id": "@id",
					    "type": "@type",
					    "title": "http://purl.org/dc/terms/title",
					    "description": "http://purl.org/dc/terms/description"
					  },
					  "@graph": [{
					     "id": "https://a.example.com/acclist",
					     "type": "acc:AccessContextList"
					    }, {
					     "id": "https://a.example.com/acclist#alpha",
					     "type": "acc:AccessContext",
					     "title": "Alpha",
					     "description": "Resources for Alpha project"
					    }, {
					     "id": "https://a.example.com/acclist#beta",
					     "type": "acc:AccessContext",
					     "title": "Beta",
					     "description": "Resources for Beta project"
					  }]
					}
					
There is no path from the acc:AccessContextList node to either of the acc:AccessContext nodes. There is an implicit containment relation of acc:AccessContext nodes in the acc:AccessContextList by virtue of these nodes being in the same information resource. However, the designers of this representation were attempting to eliminate clutter and appeal to Javascript developers, so they did not define explicit containment triples.

This user story is motivated by Linked Data and how information resources are created (e.g. via HTTP POST) or modified (e.g. via HTTP PUT). In these situations, the body of the HTTP request has an RDF content type (RDF/XML, Turtle, JSON-LD, etc.). The server typically needs to verify that the body of the request satisfies some application-specific constraints. If the request does not satisfy the constraints them it will fail the request and respond with 400 Bad Request or some similar response.

This user story draws attention to the fact that RDF content is in general a graph. The concept of RDF graph is defined in http://www.w3.org/TR/rdf11-concepts/#section-rdf-graph. A general RDF graph may not be connected and in fact disconnected RDF graphs do appear in real-world Linked Data specifications. Therefore, the output of this workgroup must support the description of constraints on general RDF graphs, connected or not.

Some of the proposed solutions (Resource Shapes, ShEx, SPIN) appear to have an implicit assumption that the only RDF graphs of interest to this workgroup are like programming language data structures in the sense that there is a distinguished root node which is the subject of triples that define either literal properties or links to other subjects, which may in turn have literal properties or links to further subjects, or so forth. The implication is that all the nodes of interest are connected to the root node. Therefore, these proposals are incapable of describing disconnected graphs. The point of this user story is to provide evidence that disconnected graphs are of interest. It also attempts to make the point that the output of this workgroup should be applicable to general RDF graphs and not just some subset of graphs that follows some popular design pattern.

The example is taken from a specification related to access control. A conformant access control service must host an access control list resource that supports HTTP GET requests. The response to an HTTP GET request have a response body whose content type is application/ld+json, i.e. JSON-LD. An example is given below. In this example, there is a distinguished root node, i.e. the node of type acc:AccessContextList, but it is not connected to the other nodes of interest, i.e. the nodes of type acc:AccessContext.

An informal specification for valid RDF graphs is as follows: "Let X be the URI of an access control list information resource. Its RDF graph must must contain X as a resource node. X must have type acc:AccessContextList. X must have a string-valued dcterms:title property and a string-valued dcterms:description property. In addition, the graph may contain zero or more other resource nodes (URIs) of type acc:AccessContext. Each of these other nodes must have a string-valued dcterms:title property and a string-valued dcterms:description property. The graph may contain other triples."

This user story does not propose that a shape language must be able to distinguish between connected and disconnected graphs.

States the requirement, that constraints over RDF graphs must be describable for both, disconnected and connected ones.

S36: Support use of inverse properties

In some cases the best RDF representation of a property-value pair may reuse a pre-existing property in which the described resource is the object and the property value is the subject. The reuse of properties is a best practice for enabling data interoperability. The fact that a pre-existing property might have the opposite direction should not be used as a justification for the creation of a new inverse property. In fact, the existence of both inverse and direct properties makes writing efficient queries more difficult since both the inverse and the direct property must be included in the query.

For example, suppose we are describing test cases and want to express the relations between test cases and the requirements that they validate. Further suppose that there is a pre-existing vocabulary for requirements that defines the property ex:isValidatedBy which asserts that the subject is validated by the object. In this case there is no need to define the inverse property ex:validates. Instead the representation of test case resources should use ex:isValidatedBy with the test case as the object and the requirement as the subject.

This situation cannot be described by the current OSLC Shapes specification because that specification has a directional bias. OSLC Shapes describe properties of a given subject node, so inverse properties cannot be used. The OSLC Shape submission proposes a possible solution. See http://www.w3.org/Submission/shapes/#inverse-properties.

For sake of simplicity, a potential constraint language shall allow the usage of properties in their inverse direction if applicable. I.e. allowing the reuse of already defined properties (in an inverse manner) in a shape, even if the node the respective shape is describing only occurs in the object position.

S37: Defining allowed/required values

The cultural heritage community has a large number of lists that control values for particular properties. These are similar to the DCMItypes, but some are quite extensive (>200 types of roles for Agents in relation to resources). There is also a concept of "authorities" which control the identities of people, places, subjects, organizations and even resources themselves. Many of these lists are centralized in major agencies (Library of Congress, Getty Art & Architecture Archive, National Library of Medicine, and national libraries throughout the world). Not all have been defined in RDF or RDF/SKOS, but those that have can be identified by their IRI domain name and pattern. Validation tools need to restrict or check usage according to the rules of the agency creating and sharing the data. Some patterns of needed validation are:

  1. must be an IRI (not a literal)
  2. must be an IRI matching this pattern (e.g. http://id.loc.gov/authorities/names/)
  3. must be an IRI matching one of >1 patterns
  4. must be a (any) literal
  5. must be one of these literals ("red" "blue" "green")
  6. must be a typed literal of this type (e.g. XML dataType)
  7. literal must have a language code
Some of these are conditional: for resources of type:A, property:P has allowed values a,b,c,f.

Requires the ability to constrain the potential values of properties of a shape.

Related to: S2,S8,S11, S23

S38: Describing and Validating Linked Data portals

A small company is specialized in the development of linked data portals. The contents of those portals are usually from statistical data that comes from Excel sheets and can easily be mapped to RDF Data Cube observations.

The company needs a way to describe the model of the RDF graphs that need to be generated from the Excel sheets which will also be published as an SPARQL endpoint. Notice that those linked data portals could contain observations which will usually be instances of qb:Observation but can contain different properties.

Some constraints could be, for example, that any observation has only one floating point value, or that any observation refers to one geographical area, one year, one indicator and one dataset. That those datasets refer to organizations and those organizations have one rdfs:label property in English, another in French, and another in Spanish, etc.

In this context, the company is looking for a solution that can be easily understood by the team of developers which are familiar work with OO programming languages, relational databases, XML technologies and some basic RDF knowledge, but they are not familiar with other semantic web technologies like SPARQL, OWL, etc.

The company also wants some solution that can be published and understood by external semantic web developers so they can easily know how to query the SPARQL endpoint.

There is also a need that the solution can be machine processable, so the contents of the linked data portal can automatically be validated.

Finally, the company would like to compare the schemas employed by the different linked data portals so they can check which are the differences between the RDF nodes that appear in those portals and they can even create new applications on top of the data aggregated by those portals.

The company would also like to promote third party companies to be able to reuse the data available in those data portals so there could be third-party applications on top of them which could, for example, visualize or compare the different observations, create faceted browsers, search engines, etc. To that end, those third party companies need some way to query the schemas available in those partals and build those applications from those schemas.

Requires the ability to constrain only specific parts of an RDF node. For example, stating that any observation has only one floating point value, regardless its other properties/values.